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LONG-TERM  GOALS 

The  long-term  goal  is  to  use  a  HYbrid  Coordinate  Ocean  Model  (HYCOM)  with  data  assimilation  in 
an  eddy-resolving,  fully  global  ocean  prediction  system  with  transition  to  the  Naval  Oceanographic 
Office  (NAVO)  at  .08°  equatorial  resolution  in  2006  and  with  .04°  resolution  by  the  end  of  the  decade. 
Equatorial  resolution  of  .08°  (.04°)  increases  from  9  km  (4.4  km)  at  the  equator  to  6  km  (3  km)  at  47° 
latitude.  The  Arctic  is  covered  by  a  bipolar  “PanAm”  grid.  At  59°N  it  matches  the  spherical  grid 
covering  the  rest  of  the  global  ocean.  The  model  will  include  shallow  water  to  a  minimum  depth  of  10- 
20  m  and  will  provide  boundary  conditions  to  finer  resolution  coastal  models  that  may  use  HYCOM  or 
a  different  ocean  model. 

OBJECTIVES 

Exploratory  development  and  evaluation  of  HYCOM  as  a  data- assimilative  next  generation  model  with 
generalized  coordinates  (hybrid  isopycnal/terrain-following  (a)/z)  and  application  of  this  model  to 
eddy-resolving  global  ocean  prediction.  First  use  the  Pacific  north  of  20°S,  the  Japan/East  Sea  (JES) 
and  the  Intra- Americas  Sea  (IAS)  as  test  beds  for  modeling,  grid  nesting  and/or  a  hierarchy  of  data 
assimilation  schemes  from  relatively  simple  to  advanced. 

APPROACH 

This  project  is  highly  collaborative  with  5  other  projects  (see  Related  Projects).  These  projects  focus 
on  other  aspects  of  the  research  such  as  ocean  dynamics,  most  aspects  of  model  development,  other 
model  domains  and  other  data  assimilation  techniques. 

HYCOM  is  designed  as  a  generalized  (hybrid  isopycnal/a/z)  coordinate  ocean  model.  It  is  isopycnal  in 
the  open  stratified  ocean,  but  reverts  to  a  terrain-following  coordinate  in  shallow  coastal  regions,  and 
to  z-level  coordinates  near  the  surface  in  the  mixed  layer.  This  generalized  vertical  coordinate 
approach  is  dynamic  in  space  and  time  via  the  layered  continuity  equation,  which  allows  a  dynamical 
transition  between  the  coordinate  types.  Like  MICOM,  HYCOM  permits  isopycnals  intersecting 
sloping  topography  by  allowing  zero  thickness  layers.  HYCOM  was  developed  from  MICOM  using 
the  theoretical  foundation  for  implementing  a  hybrid  coordinate  system  set  forth  in  Bleck  and  Boudra 
(1981)  and  Bleck  and  Benjamin  (1993). 

HYCOM  development  is  a  close  collaboration  between  Los  Alamos  National  Laboratory  (Rainer 
Bleck),  NRL  (Alan  Wallcraft)  and  the  University  of  Miami  (George  Halliwell),  where  the  person  in 
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parenthesis  is  the  lead  performer  in  each  group.  Alan  Wallcraft  is  in  charge  of  developing  and 
maintaining  the  standard  version  of  the  model,  one  that  is  scalable  and  portable  and  can  run  on  the 
latest  computer  architectures.  The  HYCOM/NOPP  project  has  the  lead  on  basic  model  development 
and  a  Common  HPC  Software  Support  Initiative  (CHSSI)  project  has  the  lead  on  scalability  and 
portability  of  the  computer  code.  Model  grid  nesting  (in  collaboration  with  U.  Miami)  is  the  main 
contribution  of  this  project  to  model  development.  HYCOM  will  be  maintained  as  a  single  source  code 
with  maximum  feasible  backward  compatibility.  The  source  code  and  documentation  will  be 
developed  to  the  standard  for  a  Navy  operational  product  and  for  a  community  ocean  model. 

Initially,  NRL  work  on  the  global  domain  will  be  at  relatively  coarse  resolution  (.72°  then  .24°  at  the 
equator)  under  the  HYCOM/NOPP  project  in  close  collaboration  with  Rainer  Bleck  (LANL)  and  Eric 
Chassignet  (U.  Miami).  This  project  will  begin  a  major  role  in  FY04  when  the  global  model  becomes 
eddy  resolving  with  an  increase  to  .08°  equatorial  (~7  km  mid-latitude)  resolution.  During  FY01-FY04 
this  project  will  work  on  development  and  evaluation  of  a  HYCOM  Pacific  model  with  up  to  .08° 
equatorial  resolution  and  Joe  Metzger  (NRF)  taking  the  lead.  The  .08°  HYCOM  Pacific  modeling  will 
use  computer  time  from  an  approved  FY02-04  DoD  High  Performance  Computing  (HPC)  Challenge 
grant  (Eric  Chassignet  lead  PI).  A  comparable  effort  is  underway  for  the  Atlantic  as  part  of  the 
HYCOM/NOPP  project  (see  separate  ONR  reports). 

The  IAS  domain  circumscribes  the  Caribbean,  Gulf  of  Mexico  and  Bahamas  region.  Tamara 
Townsend  (NRF)  has  the  lead  on  IAS  modeling  which  is  included  (1)  for  the  nesting  effort,  (2)  for  the 
opportunity  to  perform  numerous  simulations  at  .08°  resolution,  even  a  few  at  .04°,  and  (3)  as  a  high 
resolution  test  bed  for  advanced  data  assimilation  techniques.  The  HYCOM  JES  modeling  is  primarily 
the  domain  of  other  related  projects.  It  is  included  here  only  as  needed  to  support  the  data  assimilation 
effort.  The  JES  is  smaller  than  the  IAS  and  has  a  distinctly  different  mid-latitude  dynamical  regime 
(Hogan  and  Hurlburt,  2000)  that  is  more  relevant  to  the  Gulf  Stream  (Hurlburt  and  Hogan,  2000)  and 
the  Kuroshio  (Hurlburt  et  al.,  1996)  and  is  thus  an  excellent  high  resolution  test  bed  for  advanced  data 
assimilation.  Pat  Hogan  (NRF)  is  the  JES  modeler  for  this  project. 

The  main  participants  in  the  project  data  assimilation  effort  are  Gregg  Jacobs  (NRF),  Hans  Ngodock 
(associate  researcher  at  U.  Southern  Mississippi  and  former  postdoc  of  Andrew  Bennett)  and  Ole 
Martin  Smedstad  (PSI).  The  main  non-project  collaborators  are  Remy  Baraille  (SHOM  contractor  in 
France),  Geir  Evensen  (Nansen  Center  in  Norway)  and  Carlisle  Thacker  (NOAA/AOMF).  Ole  Martin 
will  take  the  lead  in  implementing  baseline  data  assimilation  techniques.  The  baseline  techniques  will 
be  used  in  comparison  to  more  advanced  techniques  for  efficiency  and  accuracy.  Gregg  and  Hans  plan 
to  work  on  HYCOM  data  assimilation  using  (1)  the  4DVAR  representer  method  with  an  adjoint  to  be 
provided  by  Remy  Baraille  and  (2)  ensemble  Kalman  filtering,  the  latter  in  collaboration  with  Geir 
Evensen.  Complete  altimeter  data  sets  from  TOPEX/POSEIDON,  ERS-2,  GFO,  JASON- 1,  and 
ENVISAT  with  additional  corrections  will  be  available  up  to  real  time;  similarly  for  MCSSTs.  In 
addition,  1/8°  mesoscale  depicting  MODAS  analyses  of  sea  surface  height  (SSH)  and  sea  surface 
temperature  (SST)  with  estimated  error  fields  will  be  available.  The  data  assimilative  model  results 
will  be  compared  with  unassimilated  data  enumerated  in  the  HYCOM/NOPP  ONR  report.  Both  sets 
will  be  used  in  model  evaluation. 
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WORK  COMPLETED 


Under  collaborative  HYCOM  projects  Alan  Wallcraft  completed  development  of  HYCOM  2.0  with  2- 
level  parallelization,  MPI  and  OpenMP,  and  FORTRAN  90  coding  style.  It  was  released  for  general 
use  on  3  July  2001.  The  mixed  layer  options  are  KPP  (Large  et  al.,  1997)  and  Kraus-Tumer. 

Joe  Metzger  is  using  HYCOM  to  model  the  Pacific  Ocean  north  of  20°S,  including  the  marginal  seas. 
He  has  run  several  .32°  climatologically-forced  10  year  simulations,  including  a  HYCOM  vs  MICOM 
mode  comparison.  In  August  2001,  he  ran  a  .16°  simulation  (-14  km  resolution  at  mid-latitudes)  for  5 
years  in  about  a  month.  Resolution  over  this  domain  will  be  increased  to  .08°  at  the  equator  for 
simulations  performed  under  an  approved  FY02-04  DoD  HPC  Challenge  project,  but  only  a  few  years 
are  planned  in  FY02. 

In  collaboration  with  U.  Miami,  Alan  Wallcraft  developed  a  one-way  nesting  capability  for  HYCOM. 
Alan  and  Tamara  Townsend  have  successfully  tested  it  using  the  .08°  HYCOM  IAS  model  with 
boundary  conditions  from  the  .32°  HYCOM  Atlantic  model  run  by  the  HYCOM/NOPP  project  (see 
separate  ONR  report).  Nested  IAS  simulations  have  been  run  up  to  a  year  as  illustrated  in  the  figure. 

Under  the  HYCOM/NOPP  project  Pat  Hogan  (NRL)  prepared  a  tar  file  with  HYCOM  2.0  bundled 
with  everything  to  continue  one  of  his  .32°  Atlantic  simulations  and  use  it  in  data  assimilation  and 
testing.  This  file  was  provided  to  Remy  Baraille  in  France  and  other  HYCOM  users.  Remy  is  working 
on  components  needed  for  the  HYCOM  data  assimilation  effort  in  this  project.  He  has  already 
implemented  the  Cooper  and  Haines  (1996)  technique  which  Ole  Martin  Smedstad  needs  for 
downward  projection  of  SSH  from  altimeter  data.  He  has  provided  it  to  Ole  Martin  and  is  now  working 
on  a  parallelized  tangent  linear  adjoint  for  HYCOM,  which  is  needed  for  the  4DVAR  representer 
method  that  Gregg  Jacobs  and  Hans  Ngodock  plan  to  use  in  this  project.  Remy  estimates  that  the 
adjoint  will  be  ready  for  general  use  by  mid  2002.  In  the  meantime  Gregg  and  Hans  plan  to  collaborate 
with  Geir  Evensen  on  ensemble  Kalman  filtering  data  assimilation,  which  does  not  use  an  adjoint. 

Hans  visited  both  Remy  and  Geir  and  acquired  Remy’s  existing  adjoint  for  MICOM  and  the  ensemble 
Kalman  filter  code  from  Geir.  Gregg,  Hans  and  Ole  Martin  plan  to  use  the  JES  domain  as  their  initial 
test  bed  for  data  assimilation  in  this  project.  Pat  Hogan  has  completed  several  JES  simulations  at  1/8°, 
1/16°  and  1/32°  resolution  mostly  under  a  separate  ONR  project  (see  related  projects  and  a  separate 
Hurlburt  and  Hogan  ONR  report). 

RESULTS 

With  the  release  of  version  2.0  via  collaborative  projects,  HYCOM  has  reached  the  maturity  required 
for  broad  application.  It  is  scalable  up  to  0(1000  cpus)  via  two  levels  of  parallelization,  either  or  both 
of  which  can  be  used,  and  it  can  run  on  computing  platforms  ranging  from  PCs  to  a  variety  of 
supercomputers  with  different  parallel  architectures.  This  was  accomplished  using  a  single  source  code 
for  all  machine  types. 

Since  Joe  Metzger  was  able  to  run  a  .16°  HYCOM  Pacific  simulation  for  5  years  in  a  month,  this 
domain  and  resolution  are  suitable  for  multiple  simulations  with  different  model  parameters  and 
atmospheric  forcing  products  and  for  experiments  with  6  hourly  interannual  atmospheric  forcing  1979- 
2001.  As  expected,  higher  resolution  is  needed  for  sufficient  inertial  flow  and  eddy  kinetic  energy  in 
the  Kuroshio  extension  east  of  about  155°E. 
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Figure  caption.  (A)  Full  domain  January  snapshot  of  sea  surface  temperature  (SST)  from  0.32° 
Atlantic  HYCOM.  (B)  May  snapshot  of  SST  in  the  Intra-Americas  Sea  (IAS)  region  of  the  same 
model  shown  in  (A).  (C)  Corresponding  SST  snapshot  from  the  nested  0.08°  IAS  HYCOM.  Both 
models  used  atmospheric  forcing  from  a  1979-1993  monthly  mean  ECMWF  climatology.  The 
Atlantic  basin  HYCOM  is  relaxed  to  the  1/8°  MODAS  climatology  at  the  open  boundaries  to  include 
the  global  thermohaline  circulation.  This  model  was  used  to  update  the  open  northern  and  eastern 
boundaries  of  the  IAS  model  (C)  with  barotropic  information  daily  and  3-D  structure  every  6  days. 


In  the  nested  IAS  domain,  boundary  conditions  are  taken  from  standard  archive  files  for  the  enclosing 
Atlantic  domain.  Thus,  the  enclosing  simulation  does  not  need  to  be  rerun  to  support  a  new  nested 
domain.  The  IAS  domain  has  a  large  open  boundary  including  a  corner.  This  makes  it  a  good  test  bed 
for  a  HYCOM  nested  grid  capability.  Comparison  of  the  .08°  IAS  and  the  .32°  Atlantic  model  results 
over  the  Caribbean  shows  the  strong  impact  of  the  resolution  increase  on  the  eddy  activity,  as 
illustrated  in  the  figure.  There  are  also  some  negative  impacts  of  the  coarse  resolution  simulation  on 
the  nested  fine  resolution  model.  The  Atlantic  model  must  have  sufficient  resolution  to  provide 
adequate  boundary  conditions  for  the  IAS  model. 

HYCOM  development  had  to  reach  a  sufficient  state  of  maturity  before  the  HYCOM  data  assimilation 
effort  could  begin  in  earnest.  That  occurred  with  the  release  of  HYCOM  2.0  on  3  July  2001.  With  the 
downward  projection  component  from  Remy  Baraille,  Ole  Martin  has  everything  needed  to  develop  a 
baseline  HYCOM  data  assimilation  capability  for  satellite  altimeter  data.  The  01  analysis  capability  for 
the  altimeter  data  is  already  in  place  and  Ole  Martin  has  developed  several  alternatives  for  mean  SSH 
to  add  to  the  altimetric  deviations  in  the  Atlantic,  IAS  and  JES  domains.  In  cases  where  a  model  mean 


4 


was  used,  a  rubber  sheeting  technique  was  used  to  reduce  any  current  pathway  errors.  Future 
improvements  in  these  means  are  anticipated  as  the  modeling  and  data  assimilation  efforts  proceed. 

The  Cooper-Haines  technique  for  downward  projection  of  SSH  has  been  used  in  MICOM  and  is  the 
first  downward  projection  technique  that  will  be  used  and  evaluated  in  HYCOM.  Later,  Ole  Martin 
will  try  the  statistical  inference  technique  in  Hurlburt  et  al.  (1990),  which  is  used  in  the  NRL  Layered 
Ocean  Model  (NLOM)  as  well  as  synthetic  temperature  and  salinity  profiles  derived  from  SSH  and 
SST  using  MODAS  statistics  based  on  historical  hydrographic  data.  MVOI  (Jim  Cummings,  NRL)  and 
an  adaptive  filter  (Remy  Baraille)  are  also  candidate  techniques.  Carlisle  Thacker’s  group  has  been 
working  on  assimilation  of  temperature  profile  data  into  HYCOM  as  part  of  the  HYCOM/NOPP 
project.  Ole  Martin  has  been  in  contact  with  Carlisle’s  group.  They  plan  to  collaborate  on 
implementing  a  scheme  which  assimilates  both  measured  and  synthetic  temperature  and  salinity 
profiles.  Using  the  JES  domain  as  a  test  bed,  the  baseline  data  assimilation  schemes  used  by  Ole 
Martin  will  be  compared  for  efficiency  and  accuracy  with  4DVAR  representer  method  and  ensemble 
Kalman  filter  results  of  Gregg  Jacobs,  Hans  Ngodock  and  collaborators. 

IMPACT/APPLICATIONS 

HYCOM  with  data  assimilation  is  planned  for  use  in  a  fully-global,  eddy-resolving  global  ocean 
prediction  system  as  noted  under  LONG-TERM  GOALS.  It  will  provide  boundary  conditions  to  finer 
resolution  coastal  models  which  may  use  HYCOM  or  a  different  ocean  model.  HYCOM  is  designed  to 
make  optimal  use  of  three  types  of  vertical  coordinate,  isopycnal,  a  and  z-level.  Isopycnals  are  the 
natural  coordinate  in  stratified  deep  water,  terrain-following  (a)  coordinates  in  shallow  water  and  z- 
levels  within  the  mixed  layer.  The  layered  continuity  equation  allows  a  dynamical  space  and  time 
varying  transition  between  the  three  coordinate  types.  HYCOM  also  permits  isopycnals  intersecting 
sloping  topography  by  allowing  zero  thickness  layers.  Therefore,  it  should  allow  accurate  transition 
between  deep  and  shallow  water,  historically  a  very  difficult  problem  for  ocean  models.  It  also  allows 
high  vertical  resolution  where  it  is  most  needed,  over  the  shelf  and  in  the  mixed  layer.  The  isopycnal 
coordinate  reduces  the  need  for  high  vertical  resolution  in  deep  water.  The  lead  PI  is  a  member  of  the 
U.S.  and  International  Steering  Teams  for  the  Global  Ocean  Data  Assimilation  Experiment  (GODAE), 
a  multinational  project  designed  to  help  justify  a  permanent  global  ocean  observing  system  by 
demonstrating  useful  near  real-time  global  ocean  products. 

TRANSITIONS 

The  6.2  project  preceding  this  one  lead  to  the  transition  (via  6.4)  of  the  first  eddy-resolving  nearly 
global  ocean  prediction  system  to  NAVO,  where  it  is  now  an  operational  product.  It  uses  NLOM  with 
1/16°  or  ~7  km  mid-latitude  resolution  and  7  layers  in  the  vertical,  including  a  bulk  mixed  layer.  It 
excludes  the  Arctic  and  most  shallow  regions.  Real-time  results  from  this  system  can  be  seen  at  the 
web  site  http://www7300.nrlssc.navy.mil/global_nlom. 

RELATED  PROJECTS 

The  multi-institutional  NOPP  project  HYCOM  Consortium  for  Data- assimilative  Ocean  Modeling  (see 
separate  ONR  reports),  a  6.1  ONR  JES  DRI  project  (see  separate  ONR  report),  6.1  EPIC,  6.1 
Thermodynamic  and  Topographic  Forcing  in  Global  Ocean  Models,  6.3  High  Fidelity  Simulation  of 
Littoral  Environments  (CHSSI)  and  6.4  ADFC  Support.  Additionally,  the  project  receives  grants  of 
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HPC  time  from  the  DoD  High  Performance  Computing  Modernization  Office.  The  lead  PI  is  a  member 
of  both  the  International  and  U.S.  GODAE  Steering  Teams. 
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